K-LM: Knowledge Augmenting in Language Models Within the Scholarly Domain

The use of superior algorithms and complex architectures in language models have successfully imparted human-like abilities to machines for specific tasks. But two significant constraints, the available training data size understanding domain-specific context, hamper pre-trained from optimal reliable performance. A potential solution tackle these limitations is equip with domain knowledge. While commonly adopted techniques Knowledge Graphs Embeddings (KGEs) inject knowledge, we provide a Language Model (K-LM) Resource Description Framework (RDF) triples directly, extracted world knowledge bases. proposed model works conjunction Generative Pretrained Transformer (GPT-2) Bidirectional Encoder Representations Transformers (BERT) uses well-defined pipeline select, categorize, filter RDF triples. In addition, introduce heuristic methods K-LM, leveraging graphs (KGs). We tested our approaches on classification task within scholarly using KGs, results show that has significantly outperformed baselines BERT each KG. Our experimental findings also help us conclude importance relevance KG used over quantity injected Also, injecting increased overall model’s accuracy, demonstrating K-LM choice adaptation solve knowledge-driven problems.